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The  purpose  of  this  paper  la  to  illustrate  some  applloatlona 
of  the  functional  equation  technique  of  the  theory  of  dynaalc 
programming  to  a  general  class  of  problems  arising  In  the  study 
of  networks,  particularly  those  arising  In  transportation 
theory .  ^  ') 
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NOTES  ON  THE  THEORY  OF  DYNARIC  PROQRAMMIHG — VII 
TRANSPORTATION  MODELS 

By 

Richard  Bellman 


§1 .  Introduction 

The  problem  of  determining  the  structure  of  networks  which 
are  optimal  with  respect  to  dlffei*ent  types  of  criteria  Is  a 
burden  shared  In  common  by  those  who  work  In  economic,  Indus¬ 
trial,  organizational,  communication,  electronic,  and  computing 
fields.  Relatively  few  of  the  variegated  aspects  of  these 
difficult  and  important  problems  have  been  sampled  to  date, 
and  none  of  these  yield  to  simple  analysis.  For  a  brief 
survey  of  these  questions  see  [l]  . 

In  this  paper  we  wish  to  make  a  slight  contribution  to  the 
general  theory  by  showing  the  applicability  of  the  functional 
equation  technique  of  the  theory  of  dynamic  programming  to  some 
problems  in  transportation  theory,  and  by  analogy  to  the  other 
fields  mentioned  above,  as  well. 

The  first  problem  we  shall  discuss  is  the  Hltchcock- 
Koopmans  transportation  problem,  cf.  0.  Dantzlg,  [ji] ,  and 
M.  M.  Flood,  (3 j  .  Next  we  shall  treat  a  multi-stage  version. 
Finally,  we  shall  treat  a  closely  related  process  arising  In 
the  studies  of  transportation  systems  by  T.  £.  Harris  and 
CO— workers .  Powerful  computational  algorithms  exist  for  the 
solutions  of  these  problems,  some  based  directly  or  indirectly 
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upon  the  simplex  method  of  Dantzlg,  and  some  upon  characteristic 
features  of  the  process,  as,  for  example,  the  "flooding 
technique"  of  A.  Boldyreff,  if\ ,  cf.  also  D.  R.  Fulkerson 
and  L.  R.  Ford,  [6]. 

The  method  *fe  present  Is  at  the  moment  not  applicable  to 
all  forms  of  the  problems  cited  above,  due  to  the  limited 
capacity  of  present-day  computing  machines.  Since  these  capa¬ 
cities  are  Increasing  with  each  year,  the  method  will  cover 
more  and  more  cases  as  time  goes  on. 

Even  at  the  present  time  this  method  seems  more  efficient 
In  treating  certain  special  cases  of  the  general  processes, 
Including  some  which  occur  in  application.  Furthermore,  the 
method  Is  equally  applicable  to  processes  where  nonlinear  and 
stochastic  elements  occur.  In  addition,  the  method  Is  useful 
In  determining  the  structure  of  optimal  policies,  the  existence 
of  "prices",  and  so  on.  These  points  will  be  discussed  In  a 
separate  paper. 

§2 .  The  Hltchcock-Kocpmans  Transportation  Problem 

A  problem  whlcn  has  been  treated  In  considerable  detail 
bj  a  number  of  authors  Is  the  following; 

"We  are  given  a  number  of  "sources".  Si,  Sa,  •'•,  S^,  ’ ’ *  *  ^ 

and  a  number  of  "sinks"  or  "terminals",  Ti,  Ta,  ••*,  T.,  **•,  T^. 

J  w 

Each  source  has  a  quantity  x^  of  resources  which  must  be 
transported  to  various  ol  the  terminals  In  such  a  way  that  the 
total  quantity  arriving  at  T^  fulfills  a  demand  y^ .  It  Is 
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Atiumed  th&t  Z  x*  •  2  y  .  Olvtn  the  dletancee,  d. 

1  i  j  J  ij 

the  eourcee  end  the  terminele,  end  aeBuming  that  the 
shipping  e  unit  quentlty  of  x^esources  between  end 
equal  to  we  wish  to  determine  the  routing  which 

the  total  cost  of  supplying  these  demands". 

One  way  of  formulating  this  problem  In  analytic 
the  following:  Let  be  the  quantity  of  iresources 
from  to  Tj .  Then 

j  *ij  ■  *1' 

I  *ij  ■  -ij  ^ 

and  the  total  cost  Is  given  by 

We  wish  to  choose  the  quantities  subject  to  the 

constraints  of  (l)j  so  as  to  minimize  (2). 

In  this  form,  the  problem  has  been  solved  numerically  by 
means  of  various  Iterative  techniques.  Of  these,  the  simplex 
method,  with  various  modifications,  seems  most  efficient. 

§3.  Dynamic  Programming  Formulation 

We  now  wish  to  formulate  the  problem  in  dynamic  programming 
terms.  To  do  this,  we  regard  the  process  as  a  multi-stage 
process  In  which  we  first  fulfill  the  demsinds  of  Tt  from  the 


between 

cost  of 

Tj  1. 

minimizes 

terms  Is 
transported 
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resourcee  of  th«  ,  then  the  demands  of  Tg  from  the  remaining 
sources  I  and  so  on. 

As  Me  have  mentioned  above »  one  advantage  of  this  formulation 
lies  in  the  fact  that  we  need  no  longer  assume  proportional 
costs,  not  valid  in  many  situations  because  of  the  existence 
of  certain  "red-tape"  or  "set-up"  costs,  and  we  can,  if  we  so 
desire,  consider  cases  where  the  cost  functions  are  stochastic. 

For  fixed  demands,  y^,  yg,  •••,  y^^,  it  is  clear  that  the 
minimum  cost  will  be  a  function  only  of  the  quantities  x^,  x^  ***,  x^i 
and  the  number  of  terminals  N.  Let  us  then  define  the  function 


(1)  *2*  *n^  -  C 

To  obtain  a  functional  equation  for  f^(x),  let  us  assume 
that  we  begin  by  allocating  the  quantities  x^^,  ***# 

to  supply  the  demand  at  T|.  Having  done  so,  we  have  a  problem 
of  precisely  the  same  type  remaining  with  N  -  1  terminals  and 
quantities  x^  -  x^^,  Xg  —  •••,  x^  —  x^^^  at  the  M  sources. 

Hence,  we  obtain  the  functional  equation 

(2)  f^(x^,  Xg,  •••,  x„)  -Min  [  Z  dj^x^j  +  ' 

rii) 

*11'  *2  “  *21'  ’*'*  *n  ~  *nl^^' 


where  the  minimum  is  over  the  region 

(3)  >  0,  x^^  -  y^. 
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for  N  >  2,  with 

(»)  ^(*1' *2"  •••>>„)- <1111*1  + 

The  sequence  fjj  Is  now  determined  recurrently  via  (2).* 

An  Important  observation  for  computational  purposes  Is 
that  the  dimension  can  always  be  reduced  from  n  to  n  —  1  since 
n  N 

(5)  1  X.  -  Z  y.. 

1-1  ^  J-1  ^ 

Hence,  for  fixed  y^,  It  Is  sufficient  to  specify  the  quantities 
*1'  *2'  *  **r>-l*  ^  write 

(6)  *2'  ^n^  -  ^2'  *n-l^* 

Consequently,  the  case  of  two  sources  reduces  to  r  one-dimensional 
problem,  three  sources  to  a  sequence  of  two-dimensional 
maxiBizatlon  problems  Involving  functions  of  two  variables, 
and  so  on. 

With  the  Inevitable  Improvement  In  the  "memory"  of  comput¬ 
ing  isachlnes,  we  shall  be  able  to  handle  the  case  of  more  and 
more  sources  In  this  fashion.  The  Important  point  Is  that  the 
utility  of  the  method  depends  only  upon  the  number  of  sources, 
while  the  number  of  terminals  may  be  exceedingly  large. 

This  method  Is  particularly  applicable  when  the  functions 
f^  are  desired  for  a  large  set  of  values  of  the  x^.  This  is 
the  case  If  the  x^  are  stochastic  variables  and  we  wish  to 


•This  Is  an  application  of  the  "principle  of  optimality",  cf.  [2] 
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determlne  the  distribution  of  the  minimum  cost,  or  If  we  can 
determine  the  distribution  of  the  x^*8  so  as  to  minimize  the 
total  cost,  assuming  that  we  have  some  degree  of  freedom  In 
assigning  the  quantities  at  the  at  the  beginning  of  the  process. 

•  A  Multi-Stage  Transportation  Problem 

Now  consider  the  situation  where  we  have  a  sequenc  of 
sources 

Ai  Ajj 

•  •  •  «  • 

(1)  ...  Bj^ 


At  the  sources  A^ ,  B^ ,  we  have  quantities  ,  and  z^ 

respectively,  and  at  ,  T^,  we  have  demands  r^,  r^,  r^ 
wnere 


(2) 


Ti  +  r^ 


r,  -  Z  X.  +  Z  y.  Z  z  . 
^  1  1  1  1 


The  process  proceeds  In  the  following  manner.  We  write 


“  ^11 

+ 

^12 

-f 

""15 

*  ^11 

+ 

^12 

4- 

^1:5 

"  ^11 

-t- 

^12 

+ 

^13 

'ij'  ^ij  - 


Oi 
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where  ^  goes  to  A^,  y  12  ®2 

^13  ^13  ^  ^13  to  C^.  Starting  with  the  new  quantities 


*2  *11 

^  yii  ^ 

^11 

at 

*2’ 

y^  x^2  yi2  ^  ^12  ®2' 

*2  *  *15 

fashion . 

+  yi3  ^ 

^13 

at 

Cj. 

the  process  continues  In  the  same 

Given  the  distances  between  the  parts,  the  problem  Is 
to  determine  the  routing  which  minimizes  tne  total  cost. 
Let 


(^)  ^Lr(*»  y>  2)  ■  the  cost  Incurred  starting  with  x  at 

y  at  and  z  at  Cj^,  and  employing 
an  optimal  policy. 

Then  y,  z)  Is  determined  as  the  solution  of  the  usual 

Hltchcock-Koopraans  transportation  problem,  and 


(5) 


tjx,  y,  z)  -  ,Mln  [Z 


(*lji 


■ff  (x  +  X  +  Y  -f  2 

^k+1'  11  ^11  ^ 


11 


(k) 


13 


where  the  minimization  Is  over  the  region  described  by  fe). 


§5.  Railway  and  Communication  Nets 

An  analogous  class  of  problems  arises  from  the  study  of 
railway  and  communication  networks.  Here  we  have  networks 
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elmllar  to  those  appearing  In  the  previous  section,  with  the 
difference  that  there  are  now  capacity  restraints  on  the  flow 
between  any  two  Junctions  and,  sometimes,  capacity  restraints 
at  the  Junctions. 

Consider  the  following  simple  network: 


A 

n 


T 


Assume  that  trains  at  A^^  and  can  be  sent  either  to  A^^^ 
or  with  the  maximum  flew  between  and  given  by 
between  A^  and  by  »  between  and  by 

and  between  and  by  2,  •••,  n  -  1, 


and  finally  the  majclmura  flow  from  A^  to  T  given  by  d^,  and  from 

B  to  T  by  d _  . 
n  2 

Starting  with  x  trains  at  A^^  and  y  trains  at  Bj^,  let 


(1) 

Clearly 


y)  ■  the  number  of  trains  arriving  at  T, 
using  an  optimal  policy. 


f^(x,  y)  -  Min  (x,  d^)  4-  Min  (x,  d^). 


(2) 
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and 

(3)  y)  -  Max  ^12  >'22^' 

where  the  maximum  1b  over  the  region 

(^)  (a)  x^p  <  X,  yp,  +  ypp  <  y 

(h)  0  ^  ^  ^11*  ^  —  ^12  —  ^12* 

®  <  ^21  -  *^21'  ^  ^  ^22  -  ^22’ 
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